Imaging systems and methods for generating auto-exposed high-dynamic-range images

ABSTRACT

Electronic devices may have camera modules that include an image sensor and processing circuitry. The image sensor may capture an image from a scene. The processing circuitry may extract image statistics and exposure levels from the image. The processing circuitry may use the image statistics and the exposure levels to generate a first exposure time, a second exposure time, and gain settings for the image sensor. The image sensor may capture additional images from the scene having long-exposure image pixel values that are captured using the first exposure time and short-exposure image pixel values that are captured using the second exposure time. The processing circuitry may generate a long-exposure image and a short-exposure image from the second image. The processing circuitry may generate auto-exposed high-dynamic-range images of the scene using the long-exposure image and the short-exposure image.

This application claims the benefit of provisional patent applicationNo. 61/597,081, filed Feb. 9, 2012 which is hereby incorporated byreference herein in its entirety.

BACKGROUND

The present invention relates to imaging devices and, more particularly,to high-dynamic-range imaging systems.

Image sensors are commonly used in electronic devices such as cellulartelephones, cameras, and computers to capture images. In a typicalarrangement, an electronic device is provided with an image sensorhaving an array of image pixels and a corresponding lens. Someelectronic devices use arrays of image sensors and arrays ofcorresponding lenses.

In certain applications, it may be desirable to capture high-dynamicrange images. While highlight and shadow detail may be lost using aconventional image sensor, highlight and shadow detail may be retainedusing image sensors with high-dynamic-range imaging capabilities.

Common high-dynamic-range (HDR) imaging systems use multiple images thatare captured by the image sensor, each image having a different exposuretime. Captured short-exposure images may retain highlight detail whilecaptured long-exposure images may retain shadow detail. In a typicaldevice, image pixel values from short-exposure images and long-exposureimages are selected to create an HDR image.

When capturing images from highly dynamic scenes (i.e., scenes having ahigh dynamic range), conventional image systems may capture images usinginappropriate exposure times for a given scene, which may causeinsufficient dynamic range capture in a combined HDR image.

It would therefore be desirable to provide improved imaging systems andexposure methods for high-dynamic-range imaging.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an illustrative imaging device that can be used to captureauto-exposed high-dynamic-range images in accordance with an embodimentof the present invention.

FIG. 2 is an illustrative diagram showing how an exposure determinationengine may be used to provide configuration data to an image sensor forcapturing auto-exposed high-dynamic-range images in accordance with anembodiment of the present invention.

FIG. 3 is a diagram showing how an exposure determination engine of thetype shown in FIG. 2 may include sub-engines for generating short andlong exposure times for capturing auto-exposed high-dynamic-range imagesin accordance with an embodiment of the present invention.

FIG. 4 is a diagram of illustrative first and second auto-exposed imageframes that may be processed by an image processing engine to form anauto-exposed high-dynamic range image in accordance with an embodimentof the present invention.

FIG. 5 is a diagram of illustrative first and second interpolatedauto-exposed image frames that may be generated from an interleavedauto-exposed image frame and passed to an image processing engine toform an auto-exposed high-dynamic range image in accordance with anembodiment of the present invention.

FIG. 6 is a diagram showing how an auto-exposure statistics engine maybe used to generate auto-exposure statistics for automatically adjustingimage exposure times in accordance with an embodiment of the presentinvention.

FIG. 7 is a diagram showing how an exposure measurement engine may beused to generate a final exposure value based on statistics generated byauto-exposure statistics engine of the type shown in FIG. 6 inaccordance with an embodiment of the present invention.

FIG. 8 is a flow chart showing how a final exposure value may begenerated by an exposure measurement engine of the type shown in FIG. 7in accordance with an embodiment of the present invention.

FIG. 9 is a diagram showing how an exposure tracking engine and anexposure setting engine may be used to generate auto-exposure gainsettings and an auto-exposed long integration time for generatingauto-exposed high-dynamic-range images in accordance with an embodimentof the present invention.

FIG. 10 is a diagram showing how an exposure ratio control engine mayuse an auto-exposed long integration time and auto-exposure statisticsto generate an auto-exposed short integration time for generatingauto-exposed high-dynamic-range images in accordance with an embodimentof the present invention.

FIG. 11 is a flow chart showing how an auto-exposed short integrationtime may be generated by an exposure ratio control engine of the typeshown in FIG. 10 for generating auto-exposed high-dynamic-range imagesin accordance with an embodiment of the present invention.

FIG. 12 is a block diagram of a processor system employing the imagesensor of FIGS. 1-11 in accordance with an embodiment of the presentinvention.

DETAILED DESCRIPTION

Imaging systems are widely used in electronic devices such as digitalcameras, computers, cellular telephones, and other electronic devices.These electronic devices may include image sensors that gather incominglight to capture an image. The image sensors may include at least oneimage pixel array. The pixels in the image pixel array may includephotosensitive elements such as photodiodes that convert the incominglight into digital data. Image sensors may have any number of pixels(e.g., hundreds or thousands or more). A typical image sensor may, forexample, have hundreds of thousands or millions of pixels (e.g.,megapixels).

FIG. 1 is a diagram of an illustrative electronic device that uses animage sensor to capture images. Electronic device 10 of FIG. 1 may be aportable electronic device such as a camera, a cellular telephone, avideo camera, or other imaging device that captures digital image data.Device 10 may include a camera module such as camera module 12 coupledto control circuitry such as processing circuitry 18. Camera module 12may be used to convert incoming light into digital image data. Cameramodule 12 may include one or more lenses 14 and one or morecorresponding image sensors 16. During image capture operations, lightfrom a scene may be focused onto each image sensor 16 using a respectivelens 14. Lenses 14 and image sensors 16 may be mounted in a commonpackage and may provide image data to processing circuitry 18.

Processing circuitry 18 may include one or more integrated circuits(e.g., image processing circuits, microprocessors, storage devices suchas random-access memory and non-volatile memory, etc.) and may beimplemented using components that are separate from image sensor 16and/or that form part of image sensor 16 (e.g., circuits that form partof an integrated circuit that controls or reads pixel signals from imagepixels in an image pixel array on image sensor 16 or an integratedcircuit within image sensor 16). Image data that has been captured byimage sensor 16 may be processed and stored using processing circuitry18. Processed image data may, if desired, be provided to externalequipment (e.g., a computer or other device) using wired and/or wirelesscommunications paths coupled to processing circuitry 18.

The dynamic range of an image may be defined as the luminance ratio ofthe brightest element in a given scene to the darkest element the givenscene. Typically, cameras and other imaging devices capture imageshaving a dynamic range that is smaller than that of real-world scenes.High-dynamic-range (HDR) imaging systems are therefore often used tocapture representative images of scenes that have regions with highcontrast, such as scenes that have portions in bright sunlight andportions in dark shadows.

An image may be considered an HDR image if it has been generated usingimaging processes or software processing designed to increase dynamicrange. As an example, HDR images may be captured by a digital camerausing a multiple integration (or multiple exposure (ME)) process. In amultiple exposure process, multiple images (sometimes referred to asimage frames) of the same scene may be captured using different exposuretimes (sometimes referred to as integration times). A short-exposureimage captured during a short integration time may better capturedetails of brightly lit portions of the scene, whereas a long-exposureimage captured during a relatively longer integration time may bettercapture details of dark portions of the scene. The short-exposure andlong-exposure images may be combined into a composite HDR image which isable to represent the brightly lit as well as the dark portions of theimage.

In another suitable arrangement, HDR images may be captured by a digitalcamera using an interleaved integration (or interleaved exposure (IE))process. In an interleaved integration process, images having rows oflong-exposure image pixel values are interleaved with rows ofshort-exposure image pixel values. The long-exposure and short-exposureimage pixel values in each interleaved image frame may be interpolatedto form interpolated values. A long-exposure image and a short-exposureimage may be generated using the long-exposure and the short-exposurevalues from the interleaved image frame and the interpolated. Thelong-exposure image and the short-exposure image may be combined toproduce a composite HDR image which is able to represent the brightlylit as well as the dark portions of the image.

During image sensor operations, the dynamic range of a scene to becaptured may vary (e.g., the dynamic range of a scene that is imaged maychange as the image sensor is pointed in different directions, as thelighting conditions of the scene change, etc.). The duration of the longexposure (or long integration) and short exposure (or short integration)when capturing HDR images using image sensor 16 may be actively adjustedduring imaging operations to allow for flexible HDR imaging of sceneshaving a variety of dynamic range conditions.

FIG. 2 is a diagram showing how image sensor 16 may use processingcircuitry such as exposure determination engine 20 to automaticallydetermine long and short exposure durations during HDR imagingoperations. Exposure determination engine 20 may be formed as a portionof processing circuitry 18 of FIG. 1, as circuitry on a common substratewith image sensor(s) 16, or may be implemented as software running onprocessing circuitry 18 or circuitry associated with image sensor(s) 16.

As shown in FIG. 2, image sensor 16 may provide image data to exposuredetermination engine 20 via path 24. Image data provided to exposuredetermination engine 20 may include images captured by image sensor 16during HDR imaging operations (e.g., short exposure image frames, longexposure image frames, interleaved image frames including rows of longexposure image pixel values and rows of short exposure image pixelvalues, etc.). Exposure determination engine 20 may process receivedimage data by collecting statistics from the received image data toautomatically determine gain settings and durations of long and shortexposures to be used by image sensor 16 for generating HDR images of agiven scene. The duration of long and short exposures may sometimes bereferred to as long exposure times and short exposure times,respectively.

Exposure determination engine 20 may provide configuration data to imagesensor 16 over path 26. The configuration data provided to image sensor16 may include configuration settings for the operation of image sensor16. For example, the configuration data may include long and shortexposure times and gain settings to be used by image sensor 16 whencapturing subsequent image data.

Image sensor 16 may capture images using the settings in theconfiguration data received from exposure determination engine 20. Forexample, image sensor 16 may capture an image using the shortintegration times, long integration times, and gain settings generatedby exposure determination engine 20. Images captured by image sensor 16using configuration data received from exposure determination engine 20may sometimes be referred to as auto-exposed image data (e.g.,auto-exposed image frames), because the auto-exposed image data iscaptured using exposure times that are automatically generated byexposure determination engine 20 in response to an imaged scene.

Auto-exposed image data generated by image sensor 16 may be passed toimage processing engine 22. Image processing engine 22 may process theauto-exposed image data to generate high-dynamic-range images. Forexample, image processing engine 22 may combine short integration pixelvalues and long integration pixel values captured by image sensor 16using the configuration data generated by exposure determination engine20 to generate high-dynamic-range images. High-dynamic-range images thatare generated by image processing engine 22 may sometimes be referred toas auto-exposed high-dynamic-range images because the long and shortexposure times used by image sensor 16 to generate auto-exposed imageframes are determined automatically using exposure determination engine20.

As shown in FIG. 4, image processing engine 22 may be used to combineshort exposure image 32 and long exposure image 34 captured using cameramodule 12 to form auto-exposed high-dynamic-range image 28 in whichimage pixel values from each of image 32 and image 34 have beenoptimally processed and combined to form the auto-exposedhigh-dynamic-range image. Short exposure image 32 may be a shortexposure image captured using an auto-exposure short integration time T2and long exposure image 34 may be captured using an auto-exposure longintegration time T1 (e.g., times T1 and T2 may be exposure timesgenerated by exposure determination engine 20). Image processing engine22 may be formed as a portion of processing circuitry 18 of FIG. 1, ascircuitry on a common substrate with image sensor(s) 16, or may beimplemented as software running on processing circuitry 18 or circuitryassociated with image sensor(s) 16.

In one suitable arrangement, image sensor 16 may use frame-sequentialexposures in which an entire image frame is captured (i.e., all pixelsin a pixel array accumulate image data) using auto-exposure longintegration time T1 before a subsequent image frame is captured usingauto-exposure short integration time T2. In another suitablearrangement, image sensor 16 uses row-sequential exposures in which aselection of pixel rows capture an image of a portion of a scene (i.e. aportion of an image frame) using auto-exposure long integration time T1and the same selection of pixel rows is used to capture a second imageof the same portion of the scene using auto-exposure short integrationtime T2 before subsequent rows are used to repeat a multiple exposureimaging process.

If desired, row-sequential exposures may be performed in which aselection of pixel rows are used to capture an image of a portion of ascene (i.e., a portion of an image frame) using auto-exposure longintegration time T1, a subsequent selection of pixel rows are used tocapture an image of a an additional portion of the scene usingauto-exposure short integration time T2 and this process is repeateduntil pixel rows have been used to capture an image having image pixelvalues with interleaved exposure times. In this type of row-sequentialimage capture operation, a subsequent image may be captured usingauto-exposure short integration time T2 for the first selection of pixelrows, auto-exposure long integration time T1 for the subsequentselection of pixel rows, and so on until all pixels rows have been usedto capture a second image having image pixel values with interleavedexposure times.

Image processing engine 22 may be used to process an image havinginterleaved long and short exposure times (sometimes referred to as aninterleaved image). Interleaved images such as interleaved image 30 ofFIG. 5 may be capturing by alternating exposure times for each adjacentrow of image pixels, every two rows of image pixels, every three rows ofimage pixels, or for larger groups of rows of image pixels. As shown inFIG. 5, image processing engine 22 may process interleaved image 30.Image pixel values for some rows of interleaved image 30 may have beencaptured using auto-exposure long integration time T1 and image pixelvalues for other rows of interleaved image 30 may have been capturedusing auto-exposure short integration time T2.

Image processing engine 22 may include sub-processing engines such asinterpolation engines. Interpolation engines in image processing engine22 may be used for generating an interpolated short-exposure image 36and an interpolated long-exposure image 38 from interleaved image 30.Interpolated images 36 and 38 may be processed by image process engine22 to produce auto-exposed high dynamic range image 28.

Auto-exposure long integration time T1 and auto-exposure shortintegration time T2 with which short exposure image 32, long exposureimage 34, and interleaved image 30 are captured by image sensor 16 maybe automatically determined using exposure determination engine 20 (FIG.2). Auto-exposure integration times T1 and T2 may be automaticallyselected by exposure determination engine 20 in response to image datacaptured from image sensor 16 (e.g., in response to dynamic rangeconditions of a given scene that is imaged). Auto-exposure integrationtimes T1 and T2 may be automatically determined by engine 20 to enableimage sensor 16 to actively capture HDR images in response to a varietyof scenes having different dynamic ranges.

Exposure determination engine 20 may determine auto-exposure integrationtimes T1 and T2 for a particular scene to be imaged by processing imagedata received from image sensor 16. As shown in FIG. 3, exposuredetermination engine 20 may include multiple sub-engines for processingimage data received from image sensor 16 to determine auto-exposureintegration times T1 and T2.

In the example of FIG. 3, exposure determination engine 20 includessub-engines such as auto-exposure statistics engine 40, exposuremeasurement engine 44, exposure tracking engine 60, exposure settingengine 62, and exposure ratio control engine 64. Auto-exposurestatistics engine 40 may receive image data from image processor 16 andmay generate auto-exposure statistics (sometimes referred to herein asimage statistics) in response to the received image data. Statisticsengine 40 may pass the auto-exposure statistics to exposure measurementengine 44 and exposure ratio control engine 64.

Exposure measurement engine 44 may determine exposure measurement dataassociated with the image data. Exposure measurement engine 44 may passthe exposure measurement data to exposure tracking engine 60. Exposuretracking engine 60 may smooth the exposure measurement data with respectto previously tracked exposure data and may generate an adjustedexposure level for the captured image data and save the current trackedexposure data. Exposure setting engine 62 may receive the adjustedexposure level from exposure tracking engine 60 and may generateauto-exposure long integration time T1 and image sensor gain settingsbased on the adjusted exposure level.

Exposure ratio control engine 64 may receive the adjusted exposure levelfrom exposure setting engine 62. Exposure ratio control engine 64 maydetermine auto-exposure short integration time T2 based on auto-exposurelong integration time T1 and the auto-exposure statistics received fromstatistics engine 40. Configuration data such as auto-exposureintegration times T1 and T2 and the image sensor gain settings may bepassed to image sensor 16 over path 26.

Exposure determination engine 20 may, for example, determineauto-exposure integration times T1 and T2 based on statistics associatedwith the image data received from image sensor 16. Exposuredetermination engine 20 may include sub-engines for identifyingstatistics associated with image data captured by image sensor 16, suchas auto-exposure statistics engine 40. As shown in FIG. 6, interleavedimage 31 captured by image sensor 16 may be provided to auto-exposurestatistics engine 40 (e.g., via path 24 of FIG. 3). Interleaved image 31may include rows that are captured by image sensor 16 using an initiallong integration time T1′ and an initial short integration time T2′.Auto-exposure statistics engine 40 may process interleaved image 31 togenerate auto-exposure statistics 42 associated with image 31. Duringimaging operations, exposure determination engine 20 may generateauto-exposure integration times T1 and T2 based on initial integrationtimes T1′ and T2′, and auto-exposure statistics 42.

Statistics engine 40 may extract a first histogram H1 associated withthe portion of interleaved image 31 that was captured using longexposure time T1′ and a second histogram H2 associated with the portionof interleaved image 31 that was captured using short exposure time T2′.Image 31 may include image data associated with multiple data channels.Histograms H1 and H2 may, for example, be collected from a luma channel,a green channel, or any other desired channel of image 31.

Statistics associated with image 31 that are extracted using engine 40may include, for example, a luma average value Y_(MEAN), a luma lowpoint value Y_(LOW) _(—) _(POINT), and two clipping percentagesPER_(CLIP1) and PER_(CLIP2) associated with histogram H2 that eachcorrespond to a different clipping threshold. Statistics engine 40 maycompute luma average value Y_(MEAN) using a logarithmic average of T1′pixel values (e.g., as measured from histogram H1). For example, lumaaverage value Y_(MEAN) may be determined using the following equation:

$\begin{matrix}{{Y_{MEAN} = {{POW}\left( {2,{\sum\limits_{i = 1}^{NUM\_ BINS}{\left( {{\log_{2}(i)}*H\; 1(i)} \right)/M}}} \right)}},} & (1)\end{matrix}$where M is the total number of image pixels in image sensor 16 that wereexposed for long exposure time T1′, where i is a given bin number ofhistogram H1, where H1(i) is the value of histogram H1 at bin i, wherelog₂(i) is a base-two logarithm of i, where NUM_BIN is the total numberof bins in histogram H1, and where POW(2, X) is a power functionequivalent to two to the power of X. In the example of equation 1,X=Σ^(NUM) ^(—) ^(BINS) _(i=1)(log₂(i)*H1(i))/M).

In another suitable arrangement, a weighted zone average may be used todetermine luma average Y_(MEAN). In this example, an image 31 havingdimensions M×N may include a number of zones of image pixels. A meanvalue ZONEAVG(j) of each zone j may have a corresponding weight W(j). Inthis example, luma average value Y_(MEAN) may be determined using thefollowing equation:

$\begin{matrix}{{Y_{MEAN} = {{POW}\left( {2,\frac{\sum\limits_{i}\left( {{\log_{2}\left( {{ZONEAVG}(j)} \right)}*{W(j)}} \right)}{\sum\limits_{i}{W(j)}}} \right)}},} & (2)\end{matrix}$where the summations are performed for each zone j over an M×N image.

Statistics engine 40 may determine luma low point value Y_(LOW) _(—)_(POINT) as a value at which the integrated area of histogram H1 reachesa selected percentage R of the total area of histogram H1. Low pointvalue Y_(LOW) _(—) _(POINT) may be determined by enforcing twoconditions on histogram H1 as given by the following equations:

$\begin{matrix}{{{\sum\limits_{i = 1}^{{Y\_ LOW}{\_ POINT}}{H\; 1(i)}} \leq {R*M}},{and}} & (3) \\{{{\sum\limits_{i = 1}^{{Y\_ LOW}{\_ POINT}}{H\; 1(i)}} > {R*M}},} & (4)\end{matrix}$where M is the total number of image pixels in image sensor 16 that wereexposed for long exposure time T1′.

Clipping percentage PER_(CLIP1) associated with histogram H2 may bedetermined as the percentage of pixel values in histogram H2 thatexceeds a pre-determined threshold value THR_(CLIP1). Pixels having apixel value that exceeds threshold value THR_(CLIP1) may sometimes bereferred to as clipping pixels. Clipping pixels may be associated withover-saturated or excessively high captured image signals. Clippingpercentage PER_(CLIP2) associated with histogram H2 may be determined asthe percentage of pixel values in histogram H2 that exceeds one-half ofthreshold value THR_(CLIP1) (e.g., that exceeds value THR_(CLIP1)/2). Ifdesired, a second clipping threshold value THR_(CLIP2) may be identifiedfor pixel values in histogram H2 (e.g., second threshold valueTHR_(CLIP2) may be different from threshold value THR_(CLIP1)/2). In thescenario where second clipping threshold value THR_(CLIP2) is definedfor pixel values in histogram H2, clipping percentage PER_(CLIP2) may bedetermined as the percentage of pixel values in histogram H2 thatexceeds threshold THR_(CLIP2). Auto-exposure statistics 42 (e.g.,Y_(MEAN), Y_(LOW) _(—) _(POINT), PER_(CLIP1), and PER_(CLIP2)) may beused by exposure determination engine 20 to determine auto-exposureintegration times T1 and T2.

The example of FIG. 6 is merely illustrative. In general, auto-exposurestatistics 42 may include any desired statistics associated imagescaptured by image sensor 16. If desired, statistics engine 40 maygenerate statistics 42 in response to any desired images captured usinginitial integration times T1′ and T2′ (e.g., images withframe-sequential exposures, row sequential exposures, interleavedexposures, etc.).

In general, the exposure level (or brightness level) of an image isrelated to the integration times used to capture the image. Exposuredetermination engine 20 may monitor the brightness of image 31 todetermine how to adjust integration times during imaging operations(e.g., to determine how to set auto-exposure integration times T1 andT2). Exposure determination engine 20 may include sub-engines formeasuring exposure levels of image 31 such as exposure measurementengine 44 (see, e.g., FIG. 3).

As shown in FIG. 7, exposure measurement engine 44 may receiveauto-exposure statistics 42 from auto-exposure statistics engine 40(FIG. 6). Exposure measurement engine 44 may determine a final exposurelevel EXP_(FINAL) of image 31 based on statistics 42. Final exposurelevel EXP_(FINAL) may characterize the exposure level of image 31.

Illustrative steps that may be performed by exposure measurement engine44 to generate final exposure level EXP_(FINAL) using auto-exposurestatistics 42 are shown in FIG. 8.

At step 50, exposure measurement engine 44 may determine intermediateexposure levels associated with image 31. For example, engine 44 maydetermine a target exposure level EXP_(TAR), a clipping exposure levelEXP_(CLIP), and a low end-point exposure level EXP_(LOWEND).

Exposure measurement engine 44 may determine target exposure levelEXP_(TAR) based on a target brightness value Y_(TAR). Target brightnessvalue Y_(TAR) may be a pre-determined brightness value that is dependentupon the brightness of the scene captured. In general, target brightnessvalue Y_(TAR) may increase as the brightness of a given scene increases.Target exposure level EXP_(TAR) may, for example, be determined usingthe following equation:EXP_(TAR)=log₂(Y _(TAR) /Y _(MEAN))  (5)where Y_(MEAN) is the luma average value (e.g., as determined byauto-exposure statistics engine 40 of FIG. 6).

Exposure measurement engine 44 may determine clipping exposure levelEXP_(CLIP) based on a clipping percentage PER_(CLIP1). For example,clipping exposure level EXP_(CLIP) may be determined using the followingequation:EXP_(CLIP) =−K*PER_(CLIP1),  (6)where PER_(CLIP1) is clipping percentage determined by statistics engine40, and where K is a proportionality constant for converting a pixelclipping percentage to a corresponding exposure level (e.g., forconverting PER_(CLIP1) to a corresponding exposure level).Proportionality constant K may, if desired, be determined fromcalibration of image sensor 16.

Exposure measurement engine 44 may determine low end exposure levelEXP_(LOWEND) based on luma low point value Y_(LOW) _(—) _(POINT). Forexample, low end exposure level EXP_(LOWEND) may be determined using thefollowing equation:EXP_(LOWEND)=log₂(Y _(LOW) _(—) _(TARGET) /Y _(LOW) _(—) _(POINT)),  (7)where Y_(LOW) _(—) _(TARGET) is a target value for a selected percentageR of the total area of histogram H1 (e.g., Y_(LOW) _(—) _(TARGET) is atarget value for low end pixels in histogram H1). Low end exposure levelEXP_(LOWEND) may be used to exclude insufficient pixel values frominclusion in final exposure level EXP_(FINAL) (e.g., pixel values thatare too low to be displayed properly using a display).

At step 52, exposure measurement engine 44 may determine a combinedexposure level EXP_(COMB) based on target exposure level EXP_(TAR) andclipping exposure level EXP_(CLIP). For example, combined exposure levelEXP_(COMB) may be determined as a linear combination (e.g., a weightedsum) of target exposure level EXP_(TAR) and clipping exposure levelEXP_(CLIP). Combined exposure level EXP_(COMB) may, for example, bedetermined using the following equation:EXP_(COMB)=(1−W _(CLIP))*EXP_(TAR) +W _(CLIP)*EXP_(CLIP),  (8)where W_(CLIP) is a weighting factor for clipping exposure levelEXP_(CLIP). Weighting factor W_(CLIP) may, for example, have a constantvalue between zero and one.

At step 54, exposure measurement engine 44 may determine whetherEXP_(LOWEND) is positive and EXP_(COMB) is negative. If EXP_(LOWEND) isgreater than zero and that EXP_(COMB) is less than zero, processing mayproceed to step 58 via path 57.

At step 58, exposure engine 44 may compute final exposure levelEXP_(FINAL) as a linear combination (e.g., a weighted sum) of low endexposure level EXP_(LOWEND) and combined exposure level EXP_(COMB). Forexample, final exposure level EXP_(FINAL) may be determined by thefollowing equation:EXP_(FINAL)=(1−W _(LOWEND))*EXP_(COMB) +W _(LOWEND)*EXP_(LOWEND),  (9)where W_(LOWEND) is any desired weighting factor for low end exposurelevel EXP_(LOWEND).

If EXP_(LOWEND) is less than zero or EXP_(COMB) is greater than zero,processing may proceed to step 56 via path 55. At step 55, exposureengine 44 may set combined exposure level EXP_(COMB) as final exposurelevel EXP_(FINAL).

Exposure determination engine 20 may include exposure processingsub-engines such as exposure tracking engine 60 and exposure settingengine 62 (FIG. 3). Exposure tracking engine 60 and exposure settingengine 62 may adjust exposure level EXP_(FINAL) to control the overallbrightness of subsequent images captured using image sensor 16 (e.g., toreduce the number of clipped pixels in the captured image, to ensuresufficient shadow detail, etc.). As shown in FIG. 9, exposure trackingengine 60 may receive final exposure level EXP_(FINAL) from exposuremeasurement engine 44. In addition, tracking engine 60 may output andsave new tracked exposure data.

Exposure tracking engine 60 may smooth the exposure levels of imagescaptured by image sensor 16 (e.g., exposure tracking engine 60 mayadjust a current exposure level to reduce or “smooth” variations betweenthe current exposure level and previous exposure levels). Trackingengine 60 may apply recursive filtering to final exposure levelEXP_(FINAL) to determine tracked exposure value EXP_(TRACK). A trackedexposure value corresponding to a current image frame such as image 31may sometimes be labeled EXP_(TRACK) ^(CUR). A tracked exposure valuecorresponding to a previous image frame (e.g., an image frame that wascaptured by image sensor 16 prior to image 31) may sometimes be labeledEXP_(TRACK) ^(PRE). Current tracked exposure level EXP_(TRACK) ^(CUR)may be determined as a linear combination (e.g., a weighted sum) ofprevious tracked exposure level EXP_(TRACK) ^(PRE) and final exposurelevel EXP_(FINAL). For example, current tracked exposure levelEXP_(TRACK) ^(CUR) may be determined using the following equation:EXP_(TRACK) ^(CUR)=(1−T)*EXP_(TRACK) ^(PRE) +T*EXP_(FINAL),  (10)where T is a tracking coefficient (or weight) that may be selected tocontrol the tracking speed of engine 60. For example, a trackingcoefficient of 1.0 may represent a maximum tracking speed of trackingengine 60.

Exposure determination engine 20 may implement an auto-exposure flagAE_(FLAG) to indicate whether to apply EXP_(FINAL) to a new exposure.Exposure tracking engine 60 may analyze current tracked exposure levelEXP_(TRACK) ^(CUR) to determine a value at which to set auto-exposureflag AE_(FLAG). For example, exposure tracking engine 60 may setauto-exposure flag AE_(FLAG) to a value of one if an adjustment isneeded for exposure (e.g., by applying EXP_(FINAL) to a new exposure)and may set flag AE_(FLAG) to a value of zero if no adjustment isneeded.

Engine 60 may compare current tracked exposure level EXP_(TRACK) ^(CUR)to a predetermined threshold THR_(A) to determine whether to adjustfinal exposure level EXP_(FINAL). For example, if the magnitude ofcurrent tracked exposure level EXP_(TRACK) ^(CUR) is greater thanthreshold THR_(A), an adjustment to final exposure level EXP_(FINAL) isneeded and auto-exposure flag AE_(FLAG) may be set to a value of one. Ifthe magnitude of current tracked exposure level EXP_(TRACK) ^(CUR) isless than or equal to threshold THR_(A), an adjustment to final exposurelevel EXP_(FINAL) is not needed and auto-exposure flag AE_(FLAG) may beset to a value of zero.

In the scenario where an adjustment to image exposure is needed (e.g.,when AE_(FLAG) is set to one), a damping factor KA may be applied tocontrol the amount of exposure used for adjusting final exposure levelEXP_(FINAL). Tracking engine 60 may determine an intermediate exposureadjustment level EXP_(ADJUST) based on final exposure level EXP_(FINAL)and the damping factor. For example, intermediate exposure adjustmentlevel EXP_(ADJUST) may be determined as a multiplicative product ofdamping factor KA and final exposure level EXP_(FINAL). If desired, amaximum adjustment exposure level MAXADJ and a minimum adjustmentexposure level MINADJ may be imposed on adjustment level EXP_(ADJUST)(e.g., MINADJ and MAXADJ may be selected to limit the range ofacceptable adjustment levels EXP_(ADJUST)). After imposing the minimumand maximum exposure limits on adjustment level EXP_(ADJUST), engine 60may determine a final exposure adjustment level EXP_(ADJUST) _(—)_(FINAL) that may be used to adjust the desired exposure level andintegration times of images captured using image sensor 16. For example,final exposure adjustment level EXP_(ADJUST) _(—) _(FINAL) may bedetermined using the following equation:

$\begin{matrix}{{E\;{XP}_{ADJUST\_ FINAL}} = \left\{ \begin{matrix}{{{{SIGN}\left( {{EX}\; P_{ADJUST}} \right)}*{MINADJ}},} & {{{if}\mspace{14mu}{{{EX}\; P_{ADJUST}}}} \leq {MINADJ}} \\{{{EX}\; P_{ADJUST}},} & {{{if}\mspace{14mu}{MINADJ}} < {{{EX}\; P_{ADJUST}}} < {MAXADJ}} \\{{{{sign}\left( {{EX}\; P_{ADJUST}} \right)}*{MAXADJ}},} & {{{{if}\mspace{14mu}{{{EX}\; P_{ADJUST}}}} \geq {MAXADJ}},}\end{matrix} \right.} & (11)\end{matrix}$where SIGN(EXP_(ADJ)) is the sign of exposure adjustment levelEXP_(ADJUST), and where |EXP_(ADJUST)| is the absolute value ofadjustment level EXP_(ADJUST).

Final exposure adjustment level EXP_(ADJUST) _(—) _(FINAL) may be passedto exposure setting engine 62. Exposure setting engine 62 may applyfinal exposure adjustment level EXP_(ADJUST) _(—) _(FINAL) to thecurrent exposure. A new exposure (i.e., after applying final adjustmentlevel EXP_(ADJUST) _(—) _(FINAL)) may be appropriately distributed amongintegration time (exposure time), analog gain, and digital gain. Inother words, exposure setting engine 62 may determine auto-exposureintegration times and gain settings to be used by image sensor 16 tocapture HDR images from a particular scene. In general, exposuredetermination engine 20 may increase exposure level by increasingintegration time up to a maximum allowed integration time and maysubsequently increasing analog and digital gain settings for imagesensor 16. Exposure determination engine 20 may decrease exposure levelby decreasing digital gain and analog gain until the digital and analoggain reach a minimum gain value, after which integration time may thenbe decreased.

Exposure setting engine 62 may distribute final exposure adjustmentlevel EXP_(ADJUST) _(—) _(FINAL) to initial long integration time T1′ togenerate auto-exposure long integration time T1. Exposure setting engine62 may provide analog and digital gain settings 64 corresponding toauto-exposure long integration time T1. Auto-exposure long integrationtime T1 and gain settings 64 may be used by image sensor 16 to generateauto-exposed high-dynamic-range images. Auto-exposed high-dynamic-rangeimages captured by image sensor 16 may be more flexible thanconventional high-dynamic-range images captured without automaticallyadjusting exposure times (e.g., auto-exposed HDR images captured byimage sensor 16 may allow for improved HDR imaging for a wide variety ofscenes and dynamic range conditions when compared to conventional HDRimaging).

Exposure determination engine 20 may include exposure processingsub-engines such as exposure ratio control engine 64 (FIG. 3). As shownin FIG. 10, exposure ratio control engine 64 may receive auto-exposurelong integration time T1 from exposure setting engine 62 and may receiveauto exposure statistics 42 from statistics engine 40 (FIG. 6). Exposureratio control engine 64 may determine auto-exposure short integrationtime T2 based on auto-exposure long integration time T1 and autoexposure statistics 42.

Auto-exposure long integration time T1 and auto-exposure shortintegration time T2 may be related by an exposure ratio EXP_(RATIO). Forexample, time T1 may be equal to the multiplicative product of exposureratio EXP_(RATIO) and time T2. Initial integration times T1′ and T2′ maybe related by an initial exposure ratio EXP_(RATIO)′ (e.g., EXP_(RATIO)′is a ratio of integration times prior to auto-exposure adjustment byexposure determination engine 20). Exposure ratio control engine 64 maydetermine exposure ratio EXP_(RATIO) based on auto exposure statistics42 and initial exposure ratio EXP_(RATIO)′. In general, a largerexposure ratio may allow image sensor 16 to capture a higher dynamicrange scene than a lower exposure ratio. Exposure ratio EXP_(RATIO) may,for example, be any suitable power of two (e.g., 1, 2, 4, 8, etc.).After determining exposure ratio EXP_(RATIO), engine 64 may applyexposure ratio EXP_(RATIO) to auto-exposure long integration time T1 togenerate auto-exposure short integration time T2. Auto-exposureintegration times T1 and T2 may be used by image sensor 16 to captureauto-exposed high-dynamic range images (e.g., auto-exposed HDR imagessuch as auto-exposed HDR image 28 of FIGS. 3 and 4).

Illustrative steps that may be performed by exposure ratio controlengine 64 to generate auto-exposure short integration time T2 usingauto-exposure statistics 42, initial exposure ratio EXP_(RATIO)′, andauto-exposure long integration time T1 are shown in FIG. 11.

A significant level used to indicate the need to change exposure ratioEXP_(RATIO) may sometimes be labeled herein as RATIO_ADJUST_LEVEL. OnceRATIO_ADJUST_LEVEL reaches a certain threshold, engine 64 may adjustexposure ratio EXP_(RATIO). RATIO_ADJUST_LEVEL may be updated for eachimage frame that is processed by exposure determination engine 20. Forexample, RATIO_ADJUST_LEVEL may be updated to monitor whether the totalnumber of clipping pixels continuously exceeds a clipping threshold. Atstep 70, engine 64 may initialize RATIO_ADJUST_LEVEL by settingRATIO_ADJUST_LEVEL to zero.

At step 72, engine 64 may determine whether auto-exposure flag AE_(FLAG)is equal to zero. If auto-exposure flag AE_(FLAG) has a non-zero value(indicating that exposure determination engine 20 is still adjustinglong exposure T1), processing may loop back to step 70 to resetRATIO_ADJUST_LEVEL to zero. If auto-exposure flag AE_(FLAG) is equal tozero (indicating that exposure determination engine 20 has finishedadjusting long exposure T1), processing may proceed to step 74.

At step 74, engine 64 may compare clipping percentage PER_(CLIP1)associated with histogram H2 (i.e., the histogram associated withinitial long integration time T2′) to clipping threshold THR_(CLIP1) andmay compare initial exposure ratio EXP_(RATIO)′ to a maximum exposureratio threshold MAX_(RATIO). Exposure ratio threshold MAX_(RATIO) may beany desired threshold that limits the exposure ratio. If clippingpercentage PER_(CLIP1) is greater than clipping threshold THR_(CLIP) andinitial exposure ratio EXP_(RATIO)′ is less than exposure ratiothreshold MAX_(RATIO), processing may proceed to step 80 via path 78.

At step 80, engine 64 may increment RATIO_ADJUST_LEVEL. Engine 64 maysubsequently compare the incremented RATIO_ADJUST_LEVEL to apredetermined ratio adjustment threshold THR_(LEVEL) (step 82).Adjustment threshold THR_(LEVEL) may be a predetermined threshold thatidentifies whether there is an excessive percentage of clipping pixelvalues for a given period of time (e.g., if RATIO_ADJUST_LEVEL isgreater than THR_(LEVEL), there may be an excessive number of clippingpixel values). If the incremented RATIO_ADJUST_LEVEL is greater thanTHR_(LEVEL), processing may proceed to step 84. If the incrementedRATIO_ADJUST_LEVEL is less than or equal to threshold THR_(LEVEL),processing may loop back to step 70 to repeat processing for theincremented RATIO_ADJUST_LEVEL.

At step 84, engine 64 may set exposure ratio EXP_(RATIO) equal to twotimes initial exposure ratio EXP_(RATIO)′. By doubling initial exposureratio EXP_(RATIO)′ to generate exposure ratio EXP_(RATIO), engine 64 mayreduce the percentage of clipping pixels in the final image. Processingmay subsequently proceed to step 86 to determine auto-exposure shortintegration time T2 based on exposure ratio EXP_(RATIO) andauto-exposure long integration time (e.g., as generated by exposuresetting engine 62 of FIG. 9). For example, engine 64 may computeauto-exposure short integration time T2 by dividing auto-exposure longintegration time T1 by exposure ratio EXP_(RATIO).

If clipping percentage PER_(CLIP) is not greater than clipping thresholdTHR_(CLIP) or initial exposure ratio EXP_(RATIO)′ is greater thanexposure ratio threshold MAX_(RATIO), processing may proceed to step 88via path 76.

At step 88, engine 64 may compare clipping percentage PER_(CLIP2)associated with histogram H2 (i.e., the histogram associated withinitial short integration time T2′) to second clipping thresholdTHR_(CLIP2) and may compare initial exposure ratio EXP_(RATIO)′ tominimum exposure ratio MIN_(RATIO). Minimum exposure ratio MIN_(RATIO)may be any desired minimum exposure ratio that limits initial exposureratio EXP_(RATIO)′ (e.g., that sets a minimum acceptable exposure ratioEXP_(RATIO)′). If clipping percentage PER_(CLIP2) is greater than secondclipping threshold THR_(CLIP2) or initial exposure ratio EXP_(RATIO)′ isless than minimum exposure ratio MIN_(RATIO), processing may loop backto step 70 via path 87. If clipping percentage PER_(CLIP2) is less thansecond clipping threshold THR_(CLIP2) and initial exposure ratioEXP_(RATIO)′ is greater than minimum exposure ratio MIN_(RATIO),processing may proceed to step 90 via path 89.

At step 90, engine 64 may decrement RATIO_ADJUST_LEVEL. Engine 64 maysubsequently compare the decremented RATIO_ADJUST_LEVEL to the negativeof predetermined ratio adjustment threshold THR_(LEVEL) (step 82). Ifthe decremented RATIO_ADJUST_LEVEL is less than −THR_(LEVEL), shortintegration time T2 may be too low (e.g., the associated pixel valuesmay be insufficient). If the decremented RATIO_ADJUST_LEVEL is less than−THR_(LEVEL), processing may proceed to step 94. If the decrementedRATIO_ADJUST_LEVEL is greater than or equal to −THR_(LEVEL), processingmay loop back to step 70.

At step 94, engine 64 may set exposure ratio EXP_(RATIO) equal to halfof initial exposure ratio EXP_(RATIO)′. By halving initial exposureratio EXP_(RATIO)′ to generate exposure ratio EXP_(RATIO), engine 64 mayimprove signal-to-noise ratio of the final image. Processing maysubsequently proceed to step 86 to determine auto-exposure shortintegration time T2 based on exposure ratio EXP_(RATIO) (as determinedat step 94) and auto-exposure long integration time T1.

Auto-exposure integration times T1 and T2 may be used to captureauto-exposed high-dynamic-range images using image sensor 16 (FIG. 2).For example, image sensor 16 may capture a short exposure image usingauto-exposure integration time T2 and a long exposure image usingauto-exposure integration time T1. Image processing engine 22 (as shownin FIG. 4) may combine the short and long exposure images captured usingauto-exposure integration times T1 and T2 to produce auto-exposedhigh-dynamic range image 28.

In another suitable arrangement, image sensor 16 may capture aninterleaved image using auto-exposure integration times T1 and T2. Imageprocessing engine 22 may subsequently produce auto-exposedhigh-dynamic-range image 28 (see FIG. 5).

The examples of FIGS. 2-11 are merely illustrative. If desired, anynumber of auto-exposure integration times may be used forhigh-dynamic-range imaging using image sensor 16. For example, three,four, or five or more auto-exposure integration times may be used. Inthe scenario where three auto-exposure integration times are used, imagesensor 16 may capture images using three integration times and exposuredetermination engine 20 may determine three auto-exposure integrationtimes based on the images captured by sensor 16.

FIG. 12 shows in simplified form a typical processor system 300, such asa digital camera, which includes an imaging device such as imagingdevice 200 (e.g., an imaging device 200 such as camera module 12 of FIG.1 employing an image processing engine such as exposure determinationengine 20 of FIGS. 2-11, and which is configured to generateauto-exposed high-dynamic-range images as described in FIGS. 1-11).Processor system 300 is exemplary of a system having digital circuitsthat could include imaging device 200. Without being limiting, such asystem could include a computer system, still or video camera system,scanner, machine vision, vehicle navigation, video phone, surveillancesystem, auto focus system, star tracker system, motion detection system,image stabilization system, and other systems employing an imagingdevice.

Processor system 300, which may be a digital still or video camerasystem, may include a lens such as lens 396 for focusing an image onto apixel array such as pixel array 201 when shutter release button 397 ispressed. Processor system 300 may include a central processing unit suchas central processing unit (CPU) 395. CPU 395 may be a microprocessorthat controls camera functions and one or more image flow functions andcommunicates with one or more input/output (I/O) devices 391 over a bussuch as bus 393. Imaging device 200 may also communicate with CPU 395over bus 393. System 300 may include random access memory (RAM) 392 andremovable memory 394. Removable memory 394 may include flash memory thatcommunicates with CPU 395 over bus 393. Imaging device 200 may becombined with CPU 395, with or without memory storage, on a singleintegrated circuit or on a different chip. Although bus 393 isillustrated as a single bus, it may be one or more buses or bridges orother communication paths used to interconnect the system components.

Various embodiments have been described illustrating systems and methodsfor generating auto-exposed HDR images of a scene using a camera modulehaving an image sensor and processing circuitry.

The image sensor may capture a first image and the processing circuitrymay generate a first exposure time and a second exposure time based onthe first captured image. The image sensor may subsequently capture asecond image having long-exposure image pixel values that are capturedusing the first exposure time and short-exposure image pixel values thatare captured using the second exposure time. The captured first imagemay include initial long-exposure image pixel values that are capturedusing an initial long-exposure time that may be different from the firstexposure time and initial short-exposure image pixel values that arecaptured using an initial short-exposure time that may be different fromthe second exposure time.

The processing circuitry may generate the first and second exposuretimes by generating statistics associated with the captured first image(e.g., image statistics associated with the initial long-exposure imagepixel values and initial short-exposure image pixel values). Thestatistics generated by the processing circuitry may include a luma meanvalue and a luma low point value of the first captured image. Ifdesired, the generated statistics may include a first and secondclipping percentages associated with the initial short-exposure imagepixel values.

The processing circuitry may determine an exposure level of the capturedfirst image based on the generated image statistics. For example, theprocessing circuitry may compute the exposure level as a linearcombination of a target exposure level and a clipping exposure levelassociated with the captured first image. The processing circuitry maygenerate a newly adjusted exposure including the first exposure time forlong exposed pixels based on the determined exposure level and imagestatistics, and may generate the second exposure time for short exposedpixels based on the first exposure time and image statistics. Ifdesired, the processing circuitry may generate image sensor gainsettings based on the exposure adjustment level to be used by the imagesensor to capture the second image.

The processing circuitry may determine an exposure ratio based on thegenerated statistics that relates the first and second exposure times.The processing circuitry may subsequently generate the second exposuretime based on the exposure ratio and the first exposure time. Theprocessing circuitry may generate a long-exposure image and ashort-exposure image from the captured second image. For example, theprocessing circuitry may interpolate the long-exposure andshort-exposure image pixel values to generate the long-exposure andshort-exposure images. The processing circuitry may combine theshort-exposure image and the long-exposure image to generate anauto-exposed high-dynamic-range image.

In another suitable arrangement, the image sensor may capture an imagehaving long-exposure image pixel values captured using the firstexposure time and an additional image having short-exposure image pixelvalues captured using the second exposure time. The processing circuitrymay subsequently generate an auto-exposed high-dynamic-range image usingthe image having long-exposure image pixel values and the image havingshort-exposure image pixel values.

The image sensor and processing circuitry for generating auto-exposedhigh-dynamic-range images may be implemented in a system that alsoincludes a central processing unit, memory, input-output circuitry, andan imaging device that further includes a pixel array, a lens forfocusing light onto the pixel array, and a data converting circuit.

The foregoing is merely illustrative of the principles of this inventionwhich can be practiced in other embodiments.

What is claimed is:
 1. A method of using an image sensor and processingcircuitry in an electronic device to generate an auto-exposedhigh-dynamic-range image, the method comprising: with the image sensor,capturing a first image; with the processing circuitry, generating afirst exposure time and a second exposure time based on the capturedfirst image, wherein generating the first exposure time and the secondexposure time comprises generating statistics associated with thecaptured first image and determining an exposure level of the capturedfirst image based on the generated statistics, and wherein determiningthe exposure level of the captured first image comprises identifying atarget exposure level and a clipping exposure level of the capturedfirst image and computing the exposure level as a linear combination ofthe target exposure level and the clipping exposure level; with theimage sensor, capturing a second image having long-exposure image pixelvalues and short-exposure image pixel values, wherein the long-exposureimage pixel values are captured using the first exposure time andwherein the short-exposure image pixel values are captured using thesecond exposure time; with the processing circuitry, generating along-exposure image and a short-exposure image from the captured secondimage; and with the processing circuitry, generating the auto-exposedhigh-dynamic-range image using the long-exposure image and theshort-exposure image.
 2. The method defined in claim 1, whereingenerating the statistics associated with the captured first imagecomprises generating a luma mean value and a luma low point value of thefirst captured image.
 3. The method defined in claim 1, whereingenerating the first exposure time and the second exposure time furthercomprises: generating an adjusted exposure level based on the determinedexposure level; and generating the first exposure time based on theadjusted exposure level.
 4. The method defined in claim 3, whereingenerating the first exposure time and the second exposure time furthercomprises: determining an exposure ratio based on the generatedstatistics; and generating the second exposure time based on thedetermined exposure ratio and the first exposure time.
 5. The methoddefined in claim 3, further comprising: with the processing circuitry,generating image sensor gain settings based on the adjusted exposurelevel.
 6. The method defined in claim 5, wherein capturing the secondimage comprises capturing the second image using the generated imagesensor gain settings.
 7. The method defined in claim 1, wherein thecaptured first image includes initial long-exposure image pixel valuesand initial short-exposure image pixel values and wherein capturing thefirst image comprises: capturing the initial long-exposure image pixelvalues using an initial long-exposure time that is different from thefirst exposure time.
 8. The method defined in claim 7, wherein capturingthe first image further comprises: capturing the initial short-exposureimage pixel values using an initial short-exposure time that isdifferent from the second exposure time.
 9. The method defined in claim1, wherein generating the long-exposure image and the short-exposureimage comprises: interpolating the long-exposure image pixel values andthe short-exposure image pixel values.
 10. A method of using an imagesensor and processing circuitry in an electronic device to generate anauto-exposed high-dynamic-range image of a scene, the method comprising:with the image sensor, capturing image data from the scene by capturinginitial short-exposure image pixel values; with the processingcircuitry, determining an auto-exposed short integration time and anauto-exposed long integration time based on the image data, whereindetermining the auto-exposed short integration time and the auto-exposedlong integration time comprises identifying first and second clippingpercentages associated with the initial short-exposure image values,wherein the first clipping percentage represents the short-exposureimage pixel values that exceed a first threshold value, and wherein thesecond clipping percentage represents the short-exposure pixel valuesthat exceed a second threshold value that is different than the firstthreshold value; with the image sensor, capturing a first image havinglong-exposure image pixel values and a second image havingshort-exposure image pixel values, wherein the short-exposure imagepixel values are captured using the auto-exposed short integration time,and wherein the long-exposure image pixel values are captured using theauto-exposed long integration time; and with the processing circuitry,generating the auto-exposed high-dynamic-range image using the first andsecond images.
 11. The method defined in claim 10, wherein capturing theimage data from the scene comprises: capturing initial long-exposureimage pixel values using a first integration time that is different fromthe auto-exposed long integration time; and capturing the initialshort-exposure image pixel values using a second integration time thatis different from the auto-exposed short integration time.
 12. Themethod defined in claim 11, wherein determining the auto-exposed shortintegration time and the auto-exposed long integration time comprises:generating image statistics from the initial long-exposure image pixelvalues and the initial short-exposure image pixel values.
 13. The methoddefined in claim 12, wherein determining the auto-exposed shortintegration time and the auto-exposed long integration time furthercomprises: determining an exposure level of the captured first imagebased on the generated statistics; generating an exposure adjustmentlevel based on the determined exposure level; and generating theauto-exposed long integration time based on the generated exposureadjustment level.
 14. The method defined in claim 10, wherein the firstthreshold value is approximately half of the second threshold value.